A THEORETICAL INVESTIGATION ON TRAINING OF PIPE-LIKE NEURAL NETWORK BENCHMARK ARCHITECTURES AND PERFORMANCE COMPARISONS OF POPULAR TRAINING ALGORITHMS

Architectures of neural networks affect the training performance artificial networks. For more consistent evaluation algorithms, hard-to-train benchmarking architectures should be used. This study introduces a benchmark network architecture, which is called pipe-like and presents analyses for popular Neural Network Backpropagation Algorithms (NNBA) well-known Metaheuristic Search (MSA). The essentially resemble an elongated fraction deep form narrowed long bottleneck learning process. Therefore, they can significantly complicate process by causing gradient vanishing problems large delays in backward propagation parameter updates throughout network. difficulties are theoretically demonstrated this considering upper bound weight according to aggregated one-neuron channels conjecture. These also contribute Baldi et al.'s channel theorem practical aspect. experiments NNBA MSA algorithms were conducted on architecture using biological dataset. Moreover, Normalized Overall Performance Scoring (NOPS) was performed criterion-based assessment overall algorithms.